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Abstract 

We investigate the nonequilibrium steady-state thermodynamics of single Brownian macro- 
molecules with inertia under feedback control in isothermal ambient fluid. With the control being 
represented by a velocity-dependent external force, we find such open systems can have a negative 
entropy production rate and we develop a mesoscopic theory consistent with the second law. We 
propose an equilibrium condition and define a class of external forces, which includes a transverse 
Lorentz force, leading to equilibrium. 
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Modern nanotechnology allows the active control of the position and velocity of nanode- 
vices by a feedback system. The system detects the positions of the nanodevices and differen- 
tiates them in manipulating a velocity-dependent external force applied to the nanodevices. 
Recently, such a velocity-dependent feedback control (VFC) has been accomplished to re- 
duce the thermal noise of a cantilever in atomic force microscopy (AFM) and dynamic 
force microscopy 3]. In [lj], a feedback system detects the velocity of the cantilever and 
reduces its thermal noise by actively changing the direction and magnitude of a force con- 
trolling the cantilever according to its motion. Another VFC experiment has been proposed 
to control and manipulate a frictional force acting on a small array of particles by limiting 
the terminal velocity of the array with a terminal attractor [3J. Even though there have been 
many experiments and theoretical models of VFC on nanodevices, their thermodynamics 
has been lacking mainly due to the ambiguity in the definition of heat dissipated from the 
nanodevices "\ 0] • This manuscript introduces a thermodynamically-consistent heat and 
provides the first rigorous theoretical thermodynamic analysis of VFC on nanodevices. 

As models for the above nanodevices, we study single macromolecules under the VFC 
in the framework of stochastic dynamics, which has been widely applied to macromolecular 
processes, e.g., ion channels p], motor proteins |7|, biochemical reactions [8|,and nanotech- 
nology. It is important to study these mesoscopic macromolecular systems |9( operating in 
nonequilibrium steady-state in terms of not only stochastic dynamics but also thermody- 
namics Q, Q] . Recently, mesoscopic nonequilibrium thermodynamics of single macro- 
molecules based on Langevin dynamics has been developed in an overdamped regime 
However, with VFC, inertia plays a fundamental role and its explicit treatment is neces- 



sary 



14j . Hence, we investigate the relationship among entropy production, detailed 



balance, and equilibrium in the presence of the inertia. 

We discover a novel feature of entropy production rate (EPR). It is shown to be composed 
of a positive entropy production rate (PEPR) and an entropy pumping rate (EPuR). The 
EPuR indicates how much entropy is pumped out of or into the macromolecule by an exter- 
nal agent manipulating a control force applied to the macromolecules. The overall entropy 
production can be negative due to the EPuR term. This provides thermodynamic origins of 
a macromolecular cooling mechanism 1 . Furthermore, our approach makes possible 
the development of macroscopic nonequilibrium steady-state thermodynamics from meso- 
scopic scale, complementary to an approach with internal degrees of freedom that develops 
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mesoscopic kinetic rules (master equations) from balance equations on hydrodynamic scale 

n 

Following the general theory of polymer dynamics [9|, the macromolecule itself (e.g., a 

II 2 

cantilever in the AFM experiment |]| ) is described by a Hamiltonian, H(x,y) = Sj^- + 
Ui n t(x), where x — (x\, x% ■ • ■ xn) and y = (yi, y2 ■ ■ ■ yN), with Xi and yi as the 3-D position- 
and momentum-vectors of the i-th hard building block of the macromolecule, respectively. 
Ui nt (x) is the internal potential of the macromolecule, e.g, Ui nt (x) = kx 2 /2 in the AFM 
experiment, with k a spring constant of the AFM cantilever. The macromolecule is confined 
in an isothermal water bath. The random collisions between solvent water molecules and 
the building blocks of the macromolecule are modeled by a Gaussian white noise. This is 
because the building block is assumed to be much larger than the water molecules and thus 



the time scales of the two can be separated 
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161 ] . Using the Einstein summation rule, the 



Langevin equation for the i-th building block of the macromolecule located in phase space 
at (Aj j2; , Aj y, Aj i2 , Yi t x? ^i,yj ^i,z) at time t is 

dX ia /dt = d Yia H(X,Y), 

dY ia /dt = -d Xia H(X, Y) + fia{Y) (1) 
+g ia (X,Y)+Vi^(t), 

where fi a {Y) is an a-component frictional force acting on the i-th building block by the sur- 
rounding water molecules and gi a {X, Y) represents velocity- and position-dependent control 
by an external agent. In the AFM experiment, the external agent is an electric feedback 
circuit detecting the motion of the cantilever and manipulating the control force g propor- 
tional to its velocity. In this experiment, / = — and g = —aV, with 7 and a positive 

tit 



constants and V a velocity, r^f^ is a fluctuation force caused by collision with water 



molecules, where ^ is Gaussian white noise with (£ia{t)£jp(~l?)) = 8(t — t')8 a p5y. Eq 
is studied in terms of its probability distribution, P(x,y,t), using Kramers equation 



18, 



19]. 



which is assumed to have a unique stationary state for our system 

Let's consider energy conservation in the Langevin dynamics. The change of internal 
energy of the macromolecule, dH(X t , Y t ), is the same as the work done on the macromolecule 
by all the external forces; i.e., dH(X t , Y t ) = (g + f + T ■ £) • dX. The work, dW(X t , Y t ), done 
on the macromolecule by the control force g, is g ■ dX. Then, we may identify the rest of the 
terms in energy balance as heat, dQ(X t ,Y t ) = — (/ + £•?) -dX [5j. This indicates how much 



heat is produced and dissipated to the surrounding water heat bath from the macromolecule 
located at (X t , Y t ) at time t during time interval dt for a stochastic process. The energy 
balance is expressed as dH = —dQ + dW. Note that (£ • V) ■ dX denotes ^ ia T{^dXjp. From 
the energy balance and Eq.(£Q), we derive an average heat dissipation rate, 

h d (t) = (dQ/dt) 

= fdxdyi-f-v-^TritFM-^Pfaytt), (2) 

where = mi8ij for any a, j3, with the mass of the i-th. building block and v ia = y ia / rrii 
its velocity. Its detailed derivation will be presented in 20}. We note that the stochastic 



integration is done in Stratonovich sense 17] 



From Eq.(J2J), we find that the frictional dissipation is related to fluctuations. It implies 
a fluctuation dissipation relation (hd{t = oo) = 0) in equilibrium and its extension in a 
nonequilibrium steady state far away from equilibrium. The average heat dissipation rate 
can be shown to vanish in equilibrium by substituting the Boltzmann distribution in Eq. © 
with the Einstein relation, T = TI = S _1 ff T /2, where we take fee = 1 and H is the 
frictional coefficient, i.e., f(y) = — H ■ v with v the velocity of the macromolecule, I a unit 
tensor, and T heat bath temperature. 

Now we apply the heat dissipation rate, Eq.©, to an entropy balance equation. 
Since the above stochastic process is Markovian, we can introduce Gibb's entropy, S = 
— J P(x, y, t) lnP(x, y, t)dxdy j^. We can connect a statistical quantity, Gibb's entropy, to 
a thermodynamic quantity, heat dissipation rate, using the Einstein relation and find the 
definite form of the EPR. We shall find the EPR is composed of two terms: one always pos- 
itive and the other whose sign depends on the external control force g(x, y). Using Kramers 
equation, the entropy balance equation is derived as 

dS{t) h d {t) 

— ^— = e p+ [t) + e pu (t) — , (3) 

with 

e p+ (t) = T _1 J II(a;, y, t) ■ J (x, y, t)dxdy, (4) 

e pu (t) = J{V y - g(x,y)}P(x,y,t)dxdy, (5) 

h d {t) = J J(x,y,t) ■ f(y)dxdy. (6) 
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We name e p+ and e pu positive entropy production rate (PEPR) and entropy pumping rate 
(EPuR), respectively, n is a thermodynamic force defined as the sum of frictional force and 
Onsager's thermodynamic force; i.e., n = — H - (v + TV y \nP). J(x,y,t) is a thermodynamic 
flux corresponding to the thermodynamic force II and is defined by — (v + TV y In P)P, i.e., 
the sum of the velocity of the macromolecule and a diffusion flow in momentum space. 



22j | . PEPR is expressed as 



Note that, as in macroscopic nonequilibrium thermodynamics 
a product of thermodynamic force and its corresponding flux. Note also that in the above 
derivation we have used a boundary condition that the macromolecule is confined in the 
heat bath. 

The PEPR is always non-negative. This implies the second law of thermodynamics. To 
obtain the physical meaning of the PEPR, let an external agent manipulate a control force 
dependent only on the position of the macromolecule, i.e., g(x). Then the EPuR vanishes. 
The entropy changes due to heat transfer and positive entropy production in nonequilibrium. 
In its stationary state, the PEPR is balanced by the heat dissipation rate hd, i.e., the 
macromolecule constantly dissipates heat to the surrounding water heat bath. Now, let the 
external agent manipulate a control force dependent on both the position and the velocity 
of the macromolecule, i.e., g(x,y). A new term EPuR appears. This term is the average of 
V y -g(x, y). When V y -g(x, y) is positive (negative), the distribution of the macromolecule at 
y in momentum space tends to be dispersed out (contracted in) by the velocity-dependent 
control force g\ i.e., the EPuR has a meaning of how much macromolecule's distribution 
in momentum space is affected by the velocity-dependent control force g. In other words, 
it describes the amount of entropy pumped out of (into) the macromolecule to (from) the 
external agent. Since the EPuR can be negative, the overall entropy production can also 
be negative. A concrete example is the AFM experiment The stationary distribution 
of the AFM cantilever is CExp[—(y 2 /2m + U int )/T e ff], where the effective temperature 
T e ff |22| is -£^T and C is a normalization constant. EPR and heat dissipation rate are 
calculated to be — ?>^a/m{^ + a) and ST^a/m^ + a), respectively j^jj]. In other words, 
heat flows from water to the macromolecule constantly! The average kinetic energy of the 
macromolecule is smaller than that of the surrounding water molecules since the control 
force acts like a frictional force on the macromolecule. The kinetic energy is transferred 
from the water heat bath to the macromolecule. The macromolecule releases the transferred 
energy to the external agent. Let the external agent be an electric circuit connected to a 
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FIG. 1: A schematic diagram of a 3-D macro-molecule immersed in a water heat bath under a con- 
trol force g = —av and frictional force / = — "fv with positive a, 7. The 2nd law of thermodynamics 
and energy conservation require dW b = dWitws ~ dQ — cLQr < dWnws- 

charging battery with infinite storage. All the transferred energy to the electric circuit can 
be stored in the battery. Now, the second law of thermodynamics seems to be violated. 
This violation stems from the calculation of the entropy of a portion of a whole system; 
the electric circuit and the macromolecule must be viewed as one whole system since they 
are strongly coupled by a feedback system that detects the velocity of the macromolecule 
and requires the control force be proportional to the velocity (See Fig. ((J)). This combined 
system acts like a refrigerator: the macromolecule and the electric circuit act as cooled air 
and an engine part of the refrigerator, respectively. If the charging battery is disconnected 
from the electric circuit, the refrigerator takes heat (—dQ) away from the water heat bath 
and dissipates it outside the electric circuit while also dissipating the work (dWRws) done 
on the refrigerator, i.e., the work needed to run the electric circuit by detecting the velocity 
of the macromolecule and requiring the velocity-dependent force. With the refrigerator 
connected to the charging battery, a portion of the work (dWjiws) done on the refrigerator 
and a portion of the heat (—dQ) transferred from the water is stored in the battery. This 
stored energy (dWu) must be less than the work done on the refrigerator to satisfy the 2nd 
law of thermodynamics. 

We propose that zero PEPR is equivalent to equilibrium; i.e., when a macromolecular 
system stops producing entropy, a control force stops working on the system on average and 
it becomes equilibrated. This can be seen from the fact that the equilibrium properties, 
such as no flux J = and no heat dissipation hd = 0, are satisfied (see Eq.(JH) and ©) 
when PEPR vanishes. In addition, we identify a class of the control force g leading to 
equilibrium and it is shown not to work on the macromolecule not only on average but also 
instantaneously if the control force is independent of the heat bath temperature. 



Macro- 
molecule 



Electric 
Circuit 
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Now, let's prove that zero PEPR guarantees Boltzmann distribution. From Eq.(J3J), 
zero PEPR is equivalent to V y lnP = —v/T. This means that P is a stationary dis- 
tribution and is factorized into momentum- and position-dependent parts, where the 
momentum-dependent part is Gaussian with variance Tm,; i.e., P ss (x,y) = P x (x)P y (y), 
where P y (y) = Exp{— T,f =1 yf /2Tmi}. Plugging this stationary distribution into Kramers 
equation, we derive — V y ■ g + v ■ {g — V x Ui nt — TV X In P x }/T = 0. This can be rewritten 
as V y ■ [{g(x, y) — V x {Ui nt (x) + TlnP x (x))}P y (y)] = 0. Finally, we derive the forms of the 
control force g leading to the stationary distribution, 

g(x,y) = Ai(x) + A 2 (x,y), (7) 

where A x (x) = V x (U int (x) + T\nP x (x)) and A 2 (x,y) is any solution satisfyi ng • 



{A 2 (x,y)P y (y)} = 0. Since the separation of g{x,y) into A 1 and A 2 is unique 25], we 
can define A\(x) as a conservative external force, i.e., U ext (x) = —Ui nt (x) — T\nP x (x) + C 
where C is a constant. Therefore, the probability distribution function P ss (x,y) becomes 
CExp[— {Tjf =1 y1 jlnii + U int (x) + U ext (x))/T] where C is a normalization constant. 

The velocity-dependent external force, A 2 (x 2 y), leads a system, in which a macromolecule 



is confined in a heat bath, to equilibrium 



A magnetic force (A 2i = qiVi x B{xi)) 



belongs to this class of forces. This class also includes other kinds of forces such as A 2i = 
Vi x B(xi)k(vf), Vi x V Vi h(vi), and (1 — mivf z /T)xi + {mi/T)v ix Vi Z Zi with arbitrary functions 
k and h. The first two forces are perpendicular to velocity so the instantaneous work 
done by these forces is zero, while the last force is not. Thus, the instantaneous work by 
external forces is not required to vanish in equilibrium. Average work, however, vanishes, 
i.e., E[v-A 2 (x,y)] = J v ■ A 2 (x,y)P eq (x,y)dxdy = T J V y ■ A 2 (x,y)P eq (x,y)dxdy = Te pu = 
from Eq.(0). In general, instantaneous work by all temperature- independent forces, A 2 , 
vanishes since A 2 is perpendicular to velocity from V y ■ A 2 — A 2 ■ v/T = 0. 



27J . De- 



Next, we investigate the relationship between zero PEPR and detailed balance 
tailed balance stems from time reversal property of the Hamiltonian governing microscopic 
dynamics and is a necessary condition on equilibrium. However, it has been proven 
that detailed balance happens to be equivalent to equilibrium in overdamped system 
What about a system with inertia under velocity- dependent control? We find that detailed 
balance is also equivalent to equilibrium. 
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Detailed balance in a stationary state is expressed as 



P(x,t\x',t )P ss (x') = P(ex > ,t\ex,t )P ss (ex), 



(8) 



where x = (x,y) and ex = (x, —y). Here, the form of the stationary distribution, P ss (x,y), 
is not specified. From Eq.fjHJ), the linear operator C of Kramers equation must satisfy 



for arbitrary fx and f'2, where dx = dxdy and we have used P ss {x) = P ss (ex) from the 
integration of Eq.(JH} over x. Then, from Eq.fjHJ), we derive a potential condition j3] 



If g is symmetric under time reversal, Eq.(|10|) is simplified as V y lnP ss = —v/T, i.e., the 
potential condition becomes equivalent to zero PEPR. Is the control force g symmetric 
under time reversal? Yes. This is because a gravitational force and an electromagnetic 
force constituting the control force are symmetric under time reversal. The detailed balance 
Eq.fjHJ), the symmetry relation in operator £ Eq.©, and potential condition Eq. fjlUj) are all 
equivalent in a Markovian system with P ss (x) = P ss (ex). Therefore, the detailed balance is 
equivalent to zero PEPR. 

In conclusion, (i) we find that EPR can be negative under velocity-dependent feedback 
control because EPuR can be negative, (ii) We show that both zero PEPR and detailed 
balance are equivalent to equilibrium. These results furnish a thermodynamically consistent 
mesoscopic theory for nonequilibrium steady-state with negative entropy pumping, (iii) We 
identify a class of external forces g(x,y) that lead to equilibrium and do not work on the 
macromolecule not only on average but also instantaneously, when this force is independent 
of the temperature of water heat bath. 
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V^lnP^x) = (rr T )- 1 • {-2H ■ v - g(ex) + g(x)}. 



(10) 
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